Layout System for Devices with Variable Display Screen Sizes and Orientations

ABSTRACT

A method for operating a data processing system having a display screen on which a GUI is displayed is disclosed. The GUI has a plurality of configurations having different aspect ratios or numbers of pixels. The method includes providing a layout description and a runtime system that generates the GUI in response to the layout description and the display configuration. The layout description defines a first container having a plurality of components to be shown in the GUI within a first container space. Each component has a component layout description within the first container. The runtime system automatically allocates the first container space depending on the display configuration, and automatically divides the first container space into plurality of component spaces. Each component is shown in a corresponding one of the component spaces. The first container space and the component spaces automatically change when the display configuration changes.

BACKGROUND

The design of graphical user interfaces (GUIs) that are to operateacross a number of different platforms having different aspect ratios,sizes, and numbers of pixels poses significant challenges. Any givendevice may have two different aspect ratios that are operative inportrait or landscape mode. The particular mode is triggered by thedevice orientation during the running of the program, and hence, theprogram must provide at least two organizations for the GUI.Furthermore, the programmer is faced with the task of designing a GUIthat must operate on a number of different devices having screen sizesthat vary from a smart phone to a large fixed computer screen.

Many techniques exist for defining a GUI. The two most commonly used arethose designed with “Code” where the programmer creates the layout for aspecific application using a programmatic interface. Visual Basic orVisual C++ are examples of this type of layout. Another type is adeclarative layout in a text form such as HTML or Microsoft XAML. Theseuse the XML structure to map into a structure in the GUI representation.

These existing techniques require the user to define the GUI in terms ofelements that have absolute size requirements in one form or another.Such GUIs do not respond well to orientation changes such as switchingbetween portrait and landscape or when switching from a tablet to aphone to a wide screen monitor. One solution to the orientation problemrequires that the designer create multiple “layouts” in which eachlayout is optimized for a different display.

SUMMARY

The present invention includes a method for operating a data processingsystem having a display screen on which a GUI is displayed. The GUI hasa plurality of configurations having different aspect ratios or numbersof pixels. The method includes providing a layout description and aruntime system that generates the GUI in response to the layoutdescription and the display configuration. The layout descriptiondefines a first container having a plurality of components to be shownin the GUI within a first container space. Each component has acomponent layout description within the first container. The runtimesystem automatically allocates the first container space depending onthe display configuration, and automatically divides the first containerspace into a plurality of component spaces. Each component is shown in acorresponding one of the component spaces. The first container space andthe component spaces automatically change when the display configurationchanges.

In one aspect of the invention, the layout description defining thefirst container indicates a vertical or horizontal organization for thecomponents, and the runtime system divides the first container spacevertically or horizontally, respectively, assigning each component to acorresponding vertical, or horizontal, space. The relative spaceassigned to each component is specified by a layout weight parameterthat can be altered from a default value by an entry in the layoutdescription. Absent a change in the layout weights of the components,the vertical or horizontal spaces are divided equally.

In another aspect of the invention, a first one of the componentsincludes text to be displayed in the GUI, and the runtime systemautomatically sets a text font for that component based on a first fontgroup specification in the layout description for that component. If twocomponents have the same font group specification, the runtime systemautomatically determines a font that ensures that both components havetheir text properly displayed, the font changing with the displayconfiguration.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A and 1B illustrate a VGroup having three buttons in portrait andlandscape mode, respectively.

FIG. 2 illustrates a three button VPanel in which the first button hastwice the weight of the remaining two buttons.

FIG. 3 illustrates the HPanel equivalent of the VPanel shown in FIGS. 1Aand 1B.

FIG. 4 illustrates the three-button panel with Button 1 allocated twicethe space of the other buttons.

FIGS. 5A and 5B illustrate a nested display in portrait and landscapemodes, respectively.

FIGS. 6A and 6B illustrate a nested display that includes a tracecomponent that displays a graphic.

FIG. 7 illustrates a display in which the size of the Button 1 componenthas been explicitly set using layout code.

FIGS. 8A and 8B illustrate a GUI that is to run on a device such as asmart phone in either portrait or landscape orientation.

FIGS. 9A and 9B illustrate a display in which the individual componentshave their fonts adjusted based on the size of the component.

FIGS. 10A and 10B illustrate a display in which the same font size isused for a number of different components.

DETAILED DESCRIPTION

In general, a GUI according to the present invention is constructed fromone or more “components”. Each component has a graphical representationand a code section that determines the behavior of the component and howthe component is displayed. In general, each component has a defaultrepresentation that automatically sizes the graphical representation forthe amount of space assigned to that component on the current display.When the display changes orientation, the amount of space for thevarious components also changes, and the graphical representations areadjusted accordingly. A collection of components can be grouped togetherin a container. The space allocated for the container is divided amongthe components in the container. There is a default space allocationthat is applied unless a different allocation is specified for thecontainer. The entire display is the highest level of container.

The manner in which the present invention provides its advantages can bemore easily understood in terms of two types of containers. In the firsttype of container, referred to as a VGroup in the following discussion,the default space allocation consists of dividing the vertical space inthe container equally among the objects in the container. It should benoted that when the display screen changes from portrait mode tolandscape mode, the available space in the container changes, thevertical space being reduced and the horizontal space being increased.Hence, each component in the container is assigned a new space when thechange in orientation takes place and the components' representationsare likewise resized.

The second type of container will be referred to as an HGroup. Thedefault space allocation in an HGroup consists of dividing thehorizontal space in the container equally among the objects in thecontainer. Again, when the display screen changes from portrait mode tolandscape mode, the available space in the container changes, thevertical space being reduced and the horizontal space being increased.

VPanels and HPanels are special types of VGroups and HGroups,respectively, in which the individual components and the panels areautomatically provided with borders that delineate the elements inquestion. Refer now to FIGS. 1A and 1B, which illustrate displays on adata processing system which includes a VGroup having three buttons inportrait and landscape mode, respectively. When the orientation changesfrom portrait to landscape, the space available for each button changessuch that the vertical height of each button is reduced and the width isincreased. The fonts are changed accordingly to fit the available space.

In one aspect of the present invention, the user defines the layout ofthe display in a declarative language such as XML. Only those items thatare not contained in the default specification of a component need beincluded in the file if the containers of the present invention areutilized. For example, the XML code for the three-button display shownin FIGS. 1A and 1B consists of

<Vpanel>   <Button text=“Button1”/>   <Button text=“Button2”/>   <Buttontext=“Button3”/> </Vpanel>

It should be noted that only the text needs to be specified, as thedisplay space is automatically divided into three vertical sectionswhich are allocated to the various buttons. The code associated with thebuttons automatically takes care of sizing the text and buttons for thecurrent display screen and orientation.

In the above-described example, the three buttons have the same size,which is the default allocation. However, the user may wish to providemore space for one of the buttons. In one aspect of the presentinvention, the user can specify the relative layout weights of thecomponents within a container. The default weight is 1. To change thedefault layout weight, a corresponding entry is made in the layout file.For example, if the user wishes to assign more space to Button 1, theXML file would be

<Vpanel>   <Button text=“Button1” layout_weight=“2”/>   <Buttontext=“Button2”/>   <Button text=“Button3”/> </Vpanel>

Refer now to FIG. 2, which illustrates a three button VPanel in whichthe first button has twice the weight of the remaining two buttons. Inthis case, Button 1 has a height that is twice that of the other twobuttons. It should be noted that the width of the button remains thesame as that of the other buttons. Since the width limits the font sizeused for the labels, Button 1 has the same font size in as the remainingbuttons.

The present invention also implements a horizontal panel, HPanel, thatprovides an analogous function to the VPanel described above. In anHPanel, the horizontal space is divided equally. HPanel is theequivalent of the VPanel shown in FIGS. 1A and 1B. The layout file forthe HPanel is as follows

<Hpanel>   <Button text=“Button1” />   <Button text=“Button2”/>  <Button text=“Button3”/> </Hpanel>

Refer now to FIG. 4, which illustrates the three-button panel withButton 1 allocated twice the space of the other buttons. Again, if morespace is to be allocated to one button, the layout weight parameter canagain be used. However, since this is an HPanel, the increased layoutweight increases the horizontal space allocated to Button 1 in the panelrather than the vertical space. The layout file for the HPanel with theincreased weight is as follows:

<Hpanel>   <Button text=“Button1” layout_weight=“2”/>   <Buttontext=“Button2”/>   <Button text=“Button3”/> </Hpanel>

It should be noted that VPanels and HPanels, as well as thecorresponding VGroup and HGroup, are themselves components that obey thesame rules as other components. Hence, the various panels and groups canbe nested within one another to provide more complex layouts without theneed to specify the exact sizes of the components.

Refer now to FIGS. 5A and 5B, which illustrate a nested display inportrait and landscape modes, respectively. These displays include anumber of nested panels. The layout description for the displays inquestion is as follows:

    <VPanel>       <HPanel layout_weight=“1.5”>       <HPanel>        <Button text=“Button1”/>         <Button text=“Button2”/>        <Button text=“Button3”/>       </HPanel>       <VPanel>        <Button text=“Button4” layout_weight=“2”/>         <Buttontext=“Button5”/>         <Button text=“Button6”/>       </VPanel>      </HPanel>  <HPanel>     <Button   text=“Button7”/>     <VPanel  layout_weight=“2”>       <Button text=“Button8”/>       <Button  text=“Button9”/>   </VPanel>       </HPanel> </VPanel>

As noted above, a VPanel and an HPanel have a distinctive border aroundtheir child components. When nested HPanel and VPanel components areused for grouping the border shading which can sometimes lead to acluttered display. The HGroup and VGroup components behave in the samemanner as HPanel and VPanel but they do not have a border. In addition,in one aspect of the invention, HGroup and VGroup have a transparentbackground while HPanel and VPanel do not. This difference isillustrated in FIGS. 6A and 6B, which illustrate a nested display thatincludes a trace component 21 that displays a graphic. FIG. 6A shows thedisplay when only VPanel and HPanel containers are used. The layout codefor FIG. 6A is as follows:

VPanel>   <Trace/>   <HPanel>     <Button text=“Button1”>     <VPanellayout_weight=“2”>       <Button text=“Button1”/>       <Buttontext=“Button2”/>     </VPanel>   </HPanel> </VPanel>

The corresponding display when HGroups and VGroups are used is shown inFIG. 6B. The layout code for FIG. 6B is as follows:

VPanel>   <Trace/>   <HGroup>     <Button text=“Button1”/>     <VGrouplayout_weight=“2”>       <Button text=“Button1”/>       <Buttontext=“Button2”/>     </VGroup>   </HGroup> </VPanel>

In the above described examples, the components have sizes that areautomatically computed. However, there are situations in which adesigner may prefer to assign specific sizes to one or more of thecomponents. In one aspect of the present invention, components can alsohave explicit sizes assigned in a manner that is separate from theweight mechanism that assigns relative sizes. Components according tothe present invention also have layout_width and layout_heightattributes that can be assigned by the designer in the layout code.

Refer now to FIG. 7, which illustrates a display in which the size ofthe Button 1 component has been explicitly set using the followinglayout code:

<HPanel>   <Button text=“Button1”     layout_width=“60dp”  layout_height=“40dp”/> <Button text=“Button2”/> <Buttontext=“Button3”/> </HPanel>

In this aspect of the invention, the layout width and height can bespecified in display pixels, px, scale independent pixels, sp, ordensity independent pixels, dp. The sp unit selects a font size based onthe display screen density as well as the user's font size preference.When this information is not available to an implementation, an sp valueis interpreted as a dp value. In general, sp is used when a size isdesired relative to user font selection preferences. The dp unit selectsa font size based on a density independent pixel. These units arerelative to a 160 dpi screen. A dp is one pixel on a 160 dpi screen buttwo pixels on a 320 dpi screen. For text, sp is preferred because itaccommodates the users' font size preference. In general, the px unitshould be avoided as it does not scale well between devices.

The introduction of a fixed size component can lead to extra space beingavailable in a container. In one aspect of the present invention, thelayout engine that is part of a runtime system library allocates spaceutilizing the layout description. The space in a container is dividedaccording to the layout_weights of the components in that container.When a component uses less than that component's share of the space, theadditional space is divided among the remaining components in proportionto their respective layout_weights. A component in which a fixed widthand height have been defined can also have a non-zero layout_weight. Inthis case, the final space allocated to the component is the sum of thespecified fixed width or height and its share of any remaining space. Ifthe designer does not wish the fixed width or height component to bestretched in this manner, a layout_weight of zero can be assigned tothat component, and hence, that component will not receive anyadditional space.

In one aspect of the present invention, components can be reduced insize beyond their intrinsic size that results from assigning fixedwidths or heights. Such reductions take place if the available space isless than the sum of the intrinsic sizes of the objects specifiedthrough the layout_width and layout_height parameters. In this case,space is taken away from each component based on its layout_weight.

Automatically sizing components that include textual material posesadditional challenges. Refer now to FIGS. 8A and 8B, which illustrate aGUI that is to run on a device such as a smart phone in either portraitor landscape orientation. In particular, the GUI includes a number ofboxes that include text. The particular GUI in FIG. 8A is running withthe device in landscape mode. When the device is rotated to portraitmode as shown in FIG. 8B, the sizes of the boxes are altered to makemore efficient use of the display space.

Unfortunately, the text within the box cannot be re-sized with the samedegree of flexibility without causing problems. The aspect ratio of thetext for a given font is normally fixed. Consider a box that has textthat just fits into the box when the screen is in the landscape mode.When the screen is rotated to portrait, the width of the box must bereduced. If the font size is not changed, as shown in FIG. 8B, theoriginal text will no longer fit in the box. Hence, the text size ischanged to accommodate the change in orientation. The amount by whichthe text font size must change to accommodate the change in orientationwill be different for different objects in the GUI.

One method for changing the font size involves adjusting the font sizeto just fit in the object when the object size changes with a change inorientation. Unfortunately, this strategy can result in differentcomponents that started with the same text size having different textsizes in the new orientation as can be seen in FIGS. 9A and 9B, whichillustrate a display in which the individual components have their fontsadjusted based on the size of the component. FIG. 9A shows the displaywith the device in landscape mode, and FIG. 9B shows the display withthe device in portrait mode. As can be seen from these figures, thisapproach can lead to a display that lacks the aesthetic qualities of theoriginal display.

In one aspect of the present invention, this problem is overcome bydefining a group of elements that all need to share the same font sizewhen the orientation or resolution changes to preserve the aestheticquality of the display. A font group is defined for these elements, andthis font is used rather than the font that would have been used by thelayout engine in the absence of the font group. Refer now to FIGS. 10Aand 10B, which illustrate a display in which the same font size is usedfor a number of different components even though one or more of thecomponents could have used a different font size. For example, theobjects shown at 71 in FIGS. 10A and 10B need to use the same font sizein each orientation. The size of the objects changes when theorientation changes; however, the font size remains constant across theobjects.

In this aspect of the present invention, a two step process is used forsetting the font size for all components of a container that are part ofthe same group. In the first step, the font size that is required to fitthe text into each component is determined. The font size is set suchthat no object has text that is cut-off in the new orientation.Different components may have different determined font sizes at thisstep.

In the second step, the minimum of the determined font sizes is thenchosen for all of the components in the font group. This ensures thatthe components have a consistent appearance while ensuring that the textis not cut-off in any of the components. Once a font is determined foreach object in a font group, the font is communicated to each object foruse by that object.

The font group to which a component is to belong can be specified in thelayout description for that component. In the case of the GUI shown inFIGS. 10A and 10B, the XML description could be as follows:

<HPanel>  <VPanel>   <Button fontGroup=”Buttons” text=”Start”/>  <Button fontGroup=“Buttons” text=“Lap”/>   <Button fontGroup-“Buttons”text=“Reset/Stop”/>  </VPanel>  <HPanel>   <Text value=“00:00:00”/> </HPanel> </HPanel>

In some cases, it may be more efficient to define the font group for a“parent” component that includes a number of “children”. In this case,the font group assignment applies to all of the children of the parent,e.g.,

<HPanel>  <VPanel fontGroup=“Buttons”>   <Button text=“Start”/>  <Button text=“Lap”/>   <Button text=“Reset/Stop”/>  </VPanel> <HPanel>   <Text value-“00:00:00”/>  </HPanel> </HPanel>

To simplify the layout code, an “implicit font group” is assigned toeach container. The implicit font group only applies to the first levelof children of the container. It ensures that the first level ofchildren will all have the same font group. An explicit font groupassignment is inherited by all of the children of the container. Theimplicit font group assignment can be avoided by providing apredetermined explicit font group assignment such as “none” or byassigning a property that is incompatible with an implicit font groupsuch as a layout_weight=0.

In another aspect of the invention, Buttons, Text Boxes, and otherprimarily textual components have a number of additional properties thatconfigure various aspects of the displayed text. The textSize propertyallows specification of a relative or absolute text size while maxCharsspecifies the maximum length of a text field.

The textSize property can specify text size in the same units asdescribed for layout size specification. In addition, a % relative sizeis defined. For text, sp is preferred because it accommodates the user'sfont size preference. The % unit is a relative automatic font size. WhentextSize is not specified for a component the layout engine will choosea font size automatically. If the font size is specified with % unitsthe resulting font size will be a percentage of the automatic font size.For example, setting textSize=50% will result in the font size being 50percent of the automatic font size.

Text size and font group can both be used in the same container. Thisallows the same automatic font group to be used for all of the elements,except that some of the elements have a reduced font size.

In some cases, the contents of some of the text fields are not known atthe time the layout is specified. For example, a text box may bepopulated by the output of a remote device and the size of the textstring will not be known until the text string actually arrives. In oneaspect of the present invention, a field property is provided thatallows the designer to inform the layout engine of the maximum size ofthe expected input. This is referred to as the MaxChars property. Whenthe MaxChars property is specified, the textual components choose a textsize based on the specified length. The layout engine will then reservespace for that number of characters. If less than that number arrives,the font size is still defined as if the defined maximum number hadarrived. This ensures that the appearance of the text does not changefrom input to input, since the text may change in length during theoperation of the GUI.

In anther aspect of the invention, a width or height can be specified tomatch that of the parent container. A width or height specified asmatch_parent informs the layout engine that the component should besized to match its parent container. That is, it will expand to fillavailable space. Children of HPanel, VPanel, HGroup, and VGroup defaultto match_parent if an explicit width or height is not specified.

In yet another aspect of the invention, a property that informs thelayout engine to size the component just big enough to accommodate itscontents is provided. A width or height can be specified as wrapcontent. This is typically used to wrap a set of fixed sized components.For example, a VPanel with the layout height=“wrap content” around afixed size Text component would result in a panel just big enough tosurround the text.

If the children in a container have either fixed or intrinsic sizes andthe designer does not wish to calculate the minimum size container thatwill contain the children, the designer can use a special version of theVPanel and HPanel containers referred to as VWrap and HWrap,respectively. These containers automatically set the height or width ofthe container to a value that is just big enough to fit all of thechildren in the container.

The positioning of a component in a container can be explicitlycontrolled using two position specification parameters. A container canspecify a layout for all of the children in the container by specifyinga value for a parameter, “gravity”. For example, if gravity=“left”, allof the components in a container will be aligned on the left boundary ofthe container. In some cases, the designer may wish to explicitlyspecify the position of one of the components in a container withoutaltering the positioning of the remaining contacts. The parameter,“layout_gravity” allows a component to define its position within theparent container. The values of gravity and layout_gravity do not affectthe size of the components. For a vertical orientation container such asa VPanel, lay_out gravity can take on the values “left”, “center”, or“right”, which result in the object being aligned with the left boundaryof the container, the center of the container, or the right boundary ofthe container, respectively. For a horizontal orientation container suchas an HPanel, lay_out gravity can take on the values “top”, “center”, or“bottom”, which result in the object being aligned with the top boundaryof the container, the center of the container, or the bottom boundary ofthe container, respectively. In addition, layout_gravity can have thevalues “center_vertical” and “center_horizontal that signal that theobject is to be vertical center of its container or the horizontalcenter of its container, respectively.

The above-described embodiments utilize containers referred to as VGroupand HGroup. However, embodiments of the present invention that utilizedifferent types of containers in which the component sizes areautomatically set can be constructed. For example, a SwipePanel isdefined in one embodiment of the present invention. The children of thispanel are “pages” in a multi-page view in which the user moves betweenpages using a gesture such as a swipe gesture or “pushing” a button. ASwipePanel is equivalent to a display that is much larger than thatprovided on the device. The allocated space on the device is a “window”on the larger display. The positioning of the window is determined bythe swipe gesture and the current position. In essence, the swipe movesthe relative position of the window and the underlying display so that adifferent portion of the underlying display appears in the window afterthe gesture.

The children of the SwipePanel may be arranged in a manner that dependson the whether the device is in portrait or landscape mode. In oneaspect of the invention, each child occupies a space on the display suchthat one child is visible at any time and fills the allocated space. Theuser changes children by performing the swipe gesture which replaces thecurrently visible child with one to the left or right of the currentchild, depending on the direction of the swipe.

The drawings in the present application show various display arrangementof components on a display of a data processing system or computer. Thedata processing system or computer is not explicitly shown as a separate“box”, since such components are well known in the art. However, it isto be understood that such displays also indicate the present of theunderlying data processing system or computer and are a representationof the data processing system or computer as well as the display.

The above-described embodiments of the present invention have beenprovided to illustrate various aspects of the invention. However, it isto be understood that different aspects of the present invention thatare shown in different specific embodiments can be combined to provideother embodiments of the present invention. In addition, variousmodifications to the present invention will become apparent from theforegoing description and accompanying drawings. Accordingly, thepresent invention is to be limited solely by the scope of the followingclaims.

What is claimed is:
 1. A method for operating a data processing systemhaving a display screen on which a GUI is displayed, said GUI having aplurality of configurations, said method comprising providing layoutdescription; and providing a runtime system that generates said GUI inresponse to said layout description and said display configuration, saidlayout description defining a first container having a plurality ofcomponents to be shown in said GUI within a first container space, eachcomponent having a component layout description within said firstcontainer, wherein said runtime system automatically allocates saidfirst container space depending on said display configuration, andautomatically divides said first container space into a plurality ofcomponent spaces, each component being shown in a corresponding one ofsaid component spaces, said first container space and said componentspaces automatically changing when said display configuration changes.2. The method of claim 1 wherein said layout description defining saidfirst container indicates a vertical or horizontal organization for saidcomponents and wherein said runtime system divides said first containerspace vertically or horizontally, respectively, assigning each componentto a corresponding vertical or horizontal space.
 3. The method of claim2 wherein said vertical or horizontal space assigned to each componentis equal to said vertical or horizontal space, respectively, of eachother component.
 4. The method of claim 2 wherein said layoutdescription specifies a layout weight for one of said components in saidfirst container and wherein said runtime system divides said vertical orhorizontal space in a manner that depends on said weight.
 5. The methodof claim 1 wherein one of said components comprises a second container.6. The method of claim 1 wherein a first one of said componentscomprises text to be displayed in said GUI and wherein said runtimesystem automatically sets a text font for that component based on afirst font group specification in said layout description for thatcomponent.
 7. The method of claim 6 wherein a second of said componentsalso has said first font group specification and wherein said runtimesystem automatically determines a font that ensures that both componentshave their text properly displayed, said font changing with said displayconfiguration.
 8. The method of claim 7 wherein text for said text ofsaid second component has a different size than said text of said firstcomponent, said size difference being determined by a text sizespecification in said layout description.
 9. The method of claim 1wherein said runtime system positions one of said components in saidfirst container based on a position specification in said layoutdescription for said component.